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Abstract 

A solution of Hilberts fourth problem lead to integral equation of the type 
generalized cosine transform. The present paper considers the solution that in- 
tegral equation by integral geometry methods and propose an inversion formula 
for reconstruction of Crofton measures from projective smooth Finsler metrics 
in R 3 . 
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1 INTRODUCTION 

A solution of Hilberts fourth problem interpreted as to construct and study all 
Finsler metrics in R 3 whose geodesies are straight lines lead to integral equation 
of the type generalized cosine transform (see [16], [20]). It is the same to ask 
about construction of all projective (continuous and linearly additive) Finsler 
metrics in R 3 (see [20],[7])- The modern approaches make it clear that the 
problem is at the basis of integral geometry, inverse problems and Finsler ge- 
ometry. First comes the A.V.Pogorelov-R.V.Ambartzumian-R.Aleksander result 
( [Tj . [B] . [15] ) stating that in two dimensional case every projective metrics can 
be obtained by Busemanns construction with a measure in the space of lines in 
the plane. This essentially solves Hilbert's fourth problem in two dimensional 
case. In the smooth case there is another approach via symplectic geometry due 
to Alvarez [5.. Considerations on the construction of projective Finsler metrics 
in two dimensional case are found by Ambartzumian and Oganyan in [5]. One 
of the results in three dimensional case says that smooth Minkowski metrics in 
R 3 are generated, via the cosine transform by signed measures in the space of 
directions in R 3 . By removing the translation invariance restriction Pogorelov 
has shown in [16 that any smooth projective Finsler metrics in R 3 is generated 
by signed measure in the space of planes. 

We denote E - the space of planes in R 3 , S 2 - the unit sphere in R 3 (the space 
of unit vectors), S u C S 2 - the oriented great circle with pole at w £ S 2 . We 
consider locally finite signed measures n in the space E, which posses densities 
with respect to the standard Euclidean motion invariant measure, i. e. (see [2]) 

fi(de) = h(e)de = h{e) dp d£ (1.1) 



where de is an element of the standard measure, (p, £) is the usual parametrization 
of a plane e: p is the distance of e from the origin O; £ G S 2 is the direction normal 



to e. By [x] we denote the bundle of planes containing the point x G R 3 . To 
define function h x on S 2 we consider the restriction of h onto [x] as a function on 
the hemisphere, since a direction completely determines a plane from [x]. Then 
we extend the restriction to S 2 by symmetry. Thus h x (£) = h{e{) for ( e S 2 , 
where G [a;] is the plane with normal £. Below h x we call the restriction of h 
onto [.?:]. 

In [16], Pogorelov showed the following result. 

Theorem 1 If H is a smooth projective Finsler metric in R 3 (H : R 3 x S 2 — > 
[0, oo ) ), then there exists a uniquely determined locally finite signed measure \x 
in the space E 7 with continuous density function h, such that 

H(x,fl) = [ 1(0,01 MO d£ for (x,Q) G R 3 x S 2 . (1.2) 
Js 2 

Here h x is the restriction of h onto [x], d£ denotes the spherical Lebesgue measure 
on S 2 . 

The measure \i is also called a Crofton measure for the Finsler metric H (see 
[2Q],[7])- Different from the approach in [16] , Theorem 1 was proved by Schneider 
[18] using expansions in spherical harmonics. Thus for smooth projective Finsler 
metrics, Pogorelov's result establish the existence of a Crofton measure, in general 
not positive. It seems that the problem of inversion of (11.21) and construction of 
the measure ll by means of H was not considered before (we refer to the survey 

m)- 

In this paper we propose an inversion formula for (|1.2[) and reconstruct h by 
given H using integral and stochastic geometry approach. 

We are interested in the inversion of the transform h — > H defined by (|1.2I) 
(i.e. we want to recover h from a given H). 

Note that, in case li is a translation invariant (t.i.) measure on E (the measure 
H can be decomposed: there exists a finite even measure m on S 2 such that 
dfi = m(d^) ■ dp = h(t;)dt; ■ dp), (jl.2p represents the cosine transform (the zonoid 
equation) 

H(Q)= [ m,0\h(0<%. (1.3) 
Js 2 

playing an important role in convexity (see |14j). If the even function H is suffi- 
ciently smooth, then (|1.3[) has a unique continuous solution h and the inversion 
formula going back to W. Blaschke is well known. A rigorous proof specifying 
differentiability assumptions was given by R.Schneider. 

Note that one cannot reduce the solution of (|1.2p to the solution of the zonoid 
equation for a fixed point, since in general that solutions are not consistent. The 
last statement follows from the following result. Let x G R 3 be a fixed point 
and fx be a measure in the space E, with continuous density function h, which is 
not translation invariant. Now we consider the translation invariant measure \x\ 
whose element has the form d[i\ = h x (£)d£ • dp. Let H and H\ are the images 
of fi and fii respectively by transform (|1.2j) . It is obvious that the restrictions of 
H(x, ■) and H\(x, ■) onto the point x coincide. 

In the present paper the problem of finding the solution of (|1.2|) we reduce 
to find the solution of an other integral equation. The concept of a flag density 
which was introduced and systematically employed by R. V. Ambartzumian, in 
[2], [3] will be of basic importance below. We repeat the definition. 

We consider the so-called directed flags (below just a flag). A flag is a triad 
/ = (x,g, e), where a; is a point in R 3 called the location of /, g is a directed 
line containing the point x, and e is an oriented plane, i.e. a plane with specified 



positive normal direction, containing g. There are two equivalent representations 
of a flag: 

/ = (aj,n,$) or / = (x,LJ,cp), 

where ft G S 2 is the spatial direction of g, $ is the planar direction in So define 
the rotation of e around g, uj £ S 2 is the normal of e, and ip is the planar direction 
in coincides with the direction of g. 

We use locally finite signed measure ^i(de) = h(e)de in the space E to define 
the following function in the space of flags T (so-called sine-square transform) (see 

mm) 

p{f)= l -( sin 2 a(£,/)M£)^ for f^T. (1.4) 

Here h x is the restriction of h onto [x]. To explain a(£, /) we write / = (x, g, e). 
Then a(£, /) is the angle between g and the trace PI e, where is the plane 
through the origin ortohogonal to e. If we represent / = [x, u>,if) then 

sin 2 a(e,/) = cos 2 (^-^), (1.5) 

where ip is the direction of the projection of £ into the plane of the flag / and 
(II. 5[) does not depend on the choice of the reference point on the plane of the 
flag /. The function p defined on the space of flags J- we call flag densities of 
measure (jl. 

If the flag density p is sufficiently often differentiable, then (|1.4[) has a unique 
continuous solution h. In [11] by author of the present paper using integral 
and stochastic geometry methods was found an inversion formula for (jl.4j) and 
reconstruct the density h of sign measure fj, in terms of its smooth flag density p. 

The problem of finding the solutions of (|1.2[) we reduce to finding the flag 
density p for which 

1 f 27r 

H(x,Q) = - p(x,n,<S>)d<I>. (1.6) 

I" JO 

and using inversion formula of (|1.4[) . 

Now we describe the inversion formula of (|1.2[) . We need to give the definitions 
of certain partial derivatives of p(f). With each directed flag we associate three 
orthogonal axes through x: by definition, axis X\ coincides with the direction of 
g; axis x^ lies within e, is orthogonal to X\ and is directed into the right half of 
e bounded by g; axis X3 is coincides with the positive normal to e. We require 
that the axes Xi,X2,x^ form a left triad. 

By p'$(f) w e denote the derivative of p at / which corresponds to positive 
rotation of / around the axis g. By definition, the positive rotation of the space 
around the axis g appears clockwise, when we look in the direction of the axis g. 

We will also denote by g^p the partial derivative in the argument x which 

is taken in the direction 77 £ S 2 . In the special cases where r\ coincides with 
directions of the axes x-i or £3 respectively, the values of will be denoted as 
p' y and p' n correspondingly. 

Our main result is the following. Let H be a smooth projective Finsler metric 
on R 3 . For every fixed x £ R 3 , we now solve the zonoid equation for the function 
H (x, ■). Since this function is of class C°°, there exists a continuous even solution 
h(x, ■) on S 2 . 

Then we put the function h(x, ■) into equation (|1.4|) instead of h x (-) and found 
the flag function p 

P(f) = P(x,v,<p) = - / cos 2 (ip~ ip)h(x,€)d£ for every (w,ip), (1.7) 
z Js 2 



which depends on x 6 R 3 . The function p is the flag density of the Crofton 
measure for the projective Finsler metric H . 

For a given plane e and a point x G e, the corresponding so-called bundle of 
flags we denote by 

< x,e >= {/ : / = (x,3,e)}. 
We consider the function so-called bundle mass at x G R 3 : 

M(x) = - [ p(x,uj,tp)dcp = l [ h(x,$d£. (1.8) 

Note that the first integral in (ll.8[) does not depend on w e S 2 . 

By /?(■) we denote the restriction of p onto < x, e >. The notation p((p) is 
reasonable since ip completely determines a flag from < x, e >. 

Theorem 2 Let H be a smooth projective Finsler metric in R 3 , that is defined 
by transform (|1.2p of a sign measure p m E iwi/i continuous density h with respect 
to the invariant measure. For a given plane e the following representation is valid 

1 /" 27r d 2 M(t) 2 /" 27r 
h(e) = M(x) + - / ^J-i^— / (^(v)+2^(^+^(^)^. (1-9) 
Z7r Jo ^ Jo 

where x is a point on e, p is defined by (|1.7|) . p(-) is the restriction of p onto 
< x, e >. 

In §3 we present the expression for p'$(ip) in terms of the derivatives of the 
function with respect to the parameters involved. 

2 FLAG DENSITIES AND MEASURES IN THE 
SPACE OF PLANES 

Equation (|1.4|) naturally emerges in Integral Geometry in the theory of recovering 
of a measure defined in the space of planes in R 3 from the known flag density 
(see 01,1111). 

It was proved in [IT] that equation (|1.4j) has the unique solution in the class of 
continuous functions and found an inversion formula. Here we present a short 
version of the proof of the formula for completeness. 

Note that, in TO] the same problem was considered for the case p is a trans- 
lation invariant measure on E (the measure p can be decomposed: there exists 
a finite even measure m on S 2 such that dp = m(d£) • dp — h(^)d^ ■ dp). In [lOj 
(see also [12j ) was obtained integral expression for the value of m for a spherical 
domain bounded by a piecewise smooth curve and and an inversion formula was 
found for a case m has a density, which first was found in [2]. 

To invert equation (|1.4j) we do the following. Let p be a smooth function 
on the space of flags that is defined by (|1.4|) of a measure p, in E possessing 
continuous density h. We consider the restriction of h onto the set of planes 
tangent to a spherical domain. Then, by integral geometry methods we find the 
integral of the restriction over a spherical disc in terms of p. Using this integrals 
we find an inversion formula for h. 

We need some results from integral geometry. Let p be a sign measure on E, 
possessing a density h with respect the invariant measure, i.e. p(de) = h(e)de. 
Given a subset A C R 3 , by [A] we denote the set of planes, that intersect A. Let 
B be a convex body with a sufficiently smooth boundary <9B. By k±, hi we denote 
the principal normal curvatures of <9B at a point s G <9B, and by fi = (s,gi,t), 
i = 1, 2 we denote the flag, where t is the plane tangent to <9B at the point s, <?.; 



is the i-th principal direction of curvature at s 6 <9B. In [11] (see also [13]) was 
found the following representation for /j,([B]). 



Theorem 3 Let p be a sign measure on E, possessing a density h with respect 
the invariant measure. For any sufficiently smooth convex body B we have the 
following representation: 



where p is the flag density of p defined by (|1.4p , ds is an area element on 9B . 

For the case p is a translation invariant measure on E the representation first 
was found in |17j . 

We will need some further definitions. By S(Q,R) we denote the sphere with 
center Q and radius R, S 2 will stand for S(0, 1). 

Assume h is the density of p. We define the following function on S(Q,R) 



where t(s) is the plane tangent to S(Q, R) at s 6 S(Q, R). 

The measure R~ 2 h*(s) ds, where ds is an area element on S(Q,R),we call the 
conditional measure on S(Q, R), generated by p (or h). 

Let A be a geodezically convex domain on S(Q,R), which is contained in some 
hemisphere. Let B £ C R 3 ,e > be the convex body bounded by S(Q,R + e), 
planes tangent to S(Q, R) at points of dA and the conical surface with the vertex 
Q and base A. In [TT] was proved the following theorem. 

Theorem 4 Let \x be a measure on E with continuous density h with respect the 
invariant measure, and let A C S(Q, R) be an open convex domain contained in 
some hemisphere. Then 



Using Theorems 3 and 4, one can calculate the values of the "conditional mea- 
sure" of various domains A C S 2 . 

Now we calculate the conditional measure of a spherical disc A C S(Q,1) of 
spherical radius a < n/2 and using this result we find an inversion formula for 
(|1.4|) . For translation invariant case the result was obtained in [TO] (see also 
[12"]). Without loss of generality, one can consider Q as the origin O. On S 2 we 
consider usual spherical coordinates (ip, v). The center of the disc chosen for the 
pole. We have: dA = {I = (<p, a) : < ip < 2ir}. 

A flag / we call a tangent flag to A at I € dA, if / is located at I, the plane of 
/ is tangent to S 2 at I e OA and the positive normal of the plane of / coincides 
with the outer normal to S 2 at / e OA, the line of / is tangent to dA at I and 
the direction on the line of / corresponds to the motion along dA, which leaves 
(locally) A on the left hand side. By < dA > we denote the set of the tangent 
flags of A. The tangent flag at I is unique and therefore p{l) is a reasonable 
notation for the value of p at the tangent flag located at I. By p(-) we denote 
the restriction of a flag function p onto < dA > . 

Theorem 5 Let p be a C 2 smooth function on the space of flags in R 3 , that is 
defined by equation (|1.4j) of a sign measure p in E with density h with respect to 




(2.1) 



h*(s)=h(t(s)) for seS(Q,R), 




(2.2) 



the invariant measure. Then for any spherical disc A C S 2 

2tt f h*(s)ds= [ M(s)ds+ [ dM ^^ ds + cosa [ M(l)d<p- 
Ja J a J a on J 

r 27t 1-2-K 1-1-K 



sin a 



p'^{l) dip — 2cosq / p(l)d(p — sina / p' y (l)dip, (2.3) 



here n is the outer normal direction to S 2 at s, dip denotes the Lebesgue measure 
on [0,2tt). 

Proof of the Theorem 5 According to Theorem 4 we have to calculate /i([B e ]) 
and /i([Bo]). In order to apply Theorem 3, instead of B e and Bo we consider 
their smooth versions B e + S(0, 5) and Bo + S(0, 6) for some 8 > 0. We have 

/i([B £ ]) = lim/i([B e + S(0,5)]), /i([Bo]) = lim/i([B + S(0,5)]). (2.4) 

d— >0 <5— s>0 

We divide the surfaces B e +S(0, 5) and B +S(O, S) into domains (part of conical 
surface and part of spherical surface) and using (|2.ip we get the expressions for 
p([B e }) and /x([Bo]) in terms of flag density. The last expressions we put into 
(12. ip and obtain (|2 .3[) . The realization of this procedure can be found in QT] (see 
also [12]). 



3 AN INVERSION FORMULA FOR (D) 

To derive an inversion formula for (|1.4[) . we express h in terms of given function 
p. Let x s e, < e, x >= {/ : /= (x,g, e)} is the bundle of flags. 

Theorem 6 Let p be a C 2 smooth function on the space of flags in R 3 , that is 
defined by transform (|1.4[) of a sign measure p in E with continuous density h. 
For a given plane e the following representation is valid 

1 r 2 ~* B 2 M(t) 2 f 2 ^ 
h(e) = M(x)+- / -—lidp— / 0/h(<P)+VM+f%y(<P))*P- (3- 1 ) 

Z7r JO °ip x K JO 

where x is a point on e, p( ) is the restriction of p onto < e,x >, M(x) is the 
bundle mass of p at x. 

Proof of the Theorem 6 Let x € e be a point. We consider a unit sphere S 2 
tangent to e at x € e and denote by A n C S 2 the spherical disk centered at x 
with the spherical radius a n . By the mean- value theorem 

h(e)= lim J- / /i*( S )ds, (3.2) 
where \A n \ is the area of A n . 

On S 2 we consider usual spherical coordinates {ip, v) with x chosen for the pole. 
We consider the restriction of M onto S 2 and by M(ip, v) we denote the value of 
the restriction at (ip, v) £ S 2 . Also, we consider the restriction of p onto < <9A >, 
where A C S 2 is the spherical disk centered at x with the spherical radius a. By 
p(<p, a) we denote the value of the restriction at I — ((p, a) £ dA. 
We represent the integral in (13.21) according to (|2.3[) and find the limit by decom- 
posing the resulting terms in powers of a n . We have 

2 

M(p, a n ) = M(p, 0) + M' a {p, 0)a n + M'JJp, 0)^ + o(a 2 ). (3.3) 



To decompose other terms we need a lemma. 



Lemma 1 For any differentiable flag function p 

^|^| Q=0 =^,0)+p>,0). (3.4) 

In its formulation p' is defined assuming that the positive normal of e is parallel 
to the outer normal to S 2 at x. Note, that p(<p) — p(ip, 0) defined on the bundle 
of flags < e, x >. 
We have 

2 

p(p, a n ) = p(tp) + (p'4v) + p' y {v))a n + (p'Uif) + 2 4y(^ + Pyy^))^f + o{a 2 n ), 

(3.5) 

= P'tiv) + ip'Uiv) + p'$y(v))an +o(a n ), (3.6) 

p' y (<p,a n ) = p' y (ip) + {p'fyifp) + Py V (<p))a n + o(a n ) . (3.7) 

After proper substitution and taking into account that 
\A n \ = 27r(l — cosa n ) ~ Tra 2 , we find 

, , 8M(x) 
h(e) = M(x) + 1 ' 



dn 

+ M -feo)^-~^ (SUM + 2 4u&) + ( 3 - 8 ) 

Here d-^fo) is the derivative of M at x G e, in the direction of normal to e which 

cm ' 

is outer normal to Sf . 
It is easy to prove that 

M»(,nn\ d 2 M(x) dM(x) 

M aa (<p,0) = -^- x (3.9) 

where 9 if ^ denotes the second derivative in the spatial direction which corre- 

sponds to the direction cp on e. We prefer to use this notation instead of M", 
because the latter would correspond to repeated differentiation on the spherical 
surface. After substitution (|3.9I) into (|3.8I) we obtain (|1.4[) . Theorem 6 is proved. 

The second integral in (|1.4[) contains the derivative To present the 

expression for p'^ in terms of the derivatives of the function with respect to 
the parameters involved we introduce on S 2 spherical coordinates in which to 
is specified by a pair (v,x), where v denotes the latitude of u>, and x denotes 
its longitude. The corresponding derivatives with respect <£> are given by the 
expressions (see. [H]): 

<p$ = -tanx • sin</> x$ = _ cos f > v * = — ^ ■ (3.10) 

cosx 

Using these formulas one can write the expression for p^. 

4 THE CONNECTION BETWEEN EQUATIONS 

(O) AND (Q) 

For a given line g and a point x € g, the corresponding bundle of flags we denote 

by 

< x, g >= {/ : / = {x,g,e)}. 



Now we integrate (|1.4[) over all bundle ol flags < x, g > with respect to Lebesgue 
measure d$ on So, where f2 is the direction of g and So is the great circle on S 2 
with pole at f2. We obtain 



— J P (x,n,$)d$ = - jf |(n,oiMO«. 



i f 2v , „ , . if 



(4.1) 



since from the definition of flag density we have (see [3]) 



I 



sin 2 a(C,/)d$ = 27r|(fi,e)|. 



(4.2) 



Note that one can obtain (|4.2|) using the cosinus theorem of spherical geometry. 

It follows from (|4.1|) if H is a smooth projective Finsler metric in R 3 and p 
is a flag density such that 



then equations (|1.2p and (|1.4I) has the same unique solution. Thus the problem 
of finding the solution of (ll.2[) we reduce to find flag density p which satisfy (|4.3I) 
for a given smooth projective Finsler metric H. 

Note that for a fixed x £ R 3 the restriction H (x, •) of a smooth projective 
Finsler metric H onto S 2 is uniquely determined by the restriction h x (-) of the 
unique solution h of (II. 2[) onto the bundle [x]. The restriction h x (-) can be found 
by solution of the zonoid equation (|1.3|) for H(-) — H(x, •). 

Also, note that for a fixed x £ R 3 the restriction p(x, ■, ■) of the flag density 
p of the sign measure p(de) = h(e)de (where h is the unique solution of (|1.3|) ) is 
uniquely determined by the restriction h x {-). 

Hence for every fixed x £ R 3 , we solve the zonoid equation for the function 
H(x 1 •). Since this function is of class C°°, there exists a continuous even solution 
h(x, •) on S 2 . Then we put the function h(x, •) into equation (|1.4p instead of h x (-) 
and found the flag density p(x, •, •) which depends on x £ R 3 . Thus we found the 
flag density p which satisfy (|4.3p . According to Theorem 6, substituting the flag 
density p into (13. 2|) we get the solution of (II. 4j) which coincides with the solution 
of (|1.2|l . Theorem 2 is proved. 
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